The Incoherence of the Economists
What is an economist? There is an easy answer: economists are social scientists who create simplified quantitative models of large-scale market phenomena. But they are more than that. Besides their scientific role, economists have achieved a social cachet that far exceeds what one might expect from a class of geeky quasi-mathematicians. In addition to being scholars, economists today often advise national leaders, occupying some of the most influential roles in government. They are commonly treated like modern sages, given column space in our most prestigious periodicals, and turned to for analysis that goes far beyond pure economics and veers into politics, culture, and even morality. They command high salaries in government, academia, and the private sector, and they even have their own Nobel Prize.
Economists and their admirers presumably believe that the respect and social prestige accorded to them is earned rather than accidental—that they are respected and honored in proportion to their numerous accomplishments and the value they can provide to society. Skeptics like me disagree, and believe that economists are overrated, overpaid, and beyond some limited cases do not contribute to society as much as they would have us believe. At any rate, given their exalted status, it seems necessary to examine more closely what value economists actually provide to society and whether it justifies their popular acclaim.
The prominent economist N. Gregory Mankiw once jokingly suggested an analogy for the value that an economist provides to society. During a talk he gave about the Great Recession, he joked that being an economist during a recession is like being an undertaker during a plague: times are sad, but business is good. Undertakers make a valuable contribution to society, but a necessarily limited one: they do not predict plagues, nor do they cure any of the victims. If economists really were analogous to undertakers, we would expect that their social standing should be greatly reduced. Merely cleaning up some of the fallout of forces that one does not understand or control is not what brings one a plum cabinet post, exorbitant salary, or Nobel Prize.
In practice, economists seem to view themselves more like doctors during a plague. They confidently claim to have solid answers about the cause of the disease (or recession) as well as how to treat and eradicate it, and make a living from convincing people of this special expertise. Mankiw himself seriously compared economists working on the economy to doctors treating a sick patient in the pages of National Affairs in 2010. Even if one accepts this analogy, it leaves ample room for both positive and negative interpretations. During the Black Death, doctors enjoyed good business, but usually provided patients only a mixture of incompetence and superstition rather than healing or real knowledge. By contrast, during some of the great successes of medical history, like the eradication of smallpox, doctors provided expertise and effort that greatly increased human well-being.
In a recent article in aeon, Alan Jay Levinovitz offered yet another analogy for what economists do for society. He wrote that economics is “the new astrology,” and that its dazzling mathematics is only a distraction from its failure at “prophecy.” To make his case, he cited numerous failures of seemingly clever economists to predict the future, make money, or avert recessions.
An apologist for economics could respond to this by saying that economics is more like the early stages of astronomy than today’s astrology. For hundreds of years, legitimate and sober-minded astronomers advocated incorrect theories like Ptolemaic geocentrism because they didn’t know any better. However, they were on the right track: after more assiduous observation and study, astronomers corrected most of their incorrect notions and have collectively given to us a mature field that is capable of understanding distant galaxies and landing a rover on Mars. Though astrology was always irredeemable, astronomy only needed time and better observational technology to become the respectable field it is today. If economics is comparable, it only needs time and better observations in order to become as powerful and as correct as astronomy.
So which is it: are economists more like astronomers, working through occasional scientific imperfections to deliver valuable insights, or are they more like astrologers, charlatans pretending to have secrets to an esoteric mystery that is really nothing more than nonsense? Are they the doctors who eradicated smallpox, tireless and altruistic in their pursuit of truth and human flourishing, or are they the plague doctors who made a living peddling myths and quackery? These are not only idle philosophical questions. The millions of dollars that U.S. taxpayers spend annually on full-time government research economists and grants to academic economic research deserve to be properly justified or, if necessary, rerouted.
In this essay, I will attempt to quantify the performance of the economics profession over the last several decades. I will argue that the data suggests that economics has progressed very little for many years, and performs especially poorly around the time of recessions, when it would be most important for economists to perform their duties well. Because of this, I will also propose an adjusted and more modest role for economists in public life.
Case Study: Macroeconomic Prediction
To evaluate the performance of economists, it may be helpful to explore some data from the Survey of Professional Forecasters (SPF), a quarterly survey that has been conducted by the Federal Reserve Bank of Philadelphia since 1968. The survey is simple: the Federal Reserve Bank contacts professional economic forecasters and asks them to predict future values of a variety of macroeconomic indicators, including unemployment and nominal GDP. Surveyed professionals make predictions for the near future (including the current quarter) as well as the more distant future (up to about four quarters in advance). Anonymized records of predicted values and actual values since 1968 are available for free on the Philadelphia Federal Reserve Bank’s website.
In the SPF data, measuring the accuracy of a forecast is straightforward—it is simply the difference between a predicted value and the corresponding actual value, where the differences closest to 0 represent the best performance and the differences furthest from 0 represent the worst. An example data point might come from a forecaster who was surveyed during the second quarter of 1975, and was asked to make predictions about the final measured unemployment in the second quarter of 1976. If the forecaster guessed 4.5 percent unemployment but the actual realized value was 6.0 percent (for example), his absolute error was 1.5 percentage points. If another forecaster guessed 7.5 percent for the same quarter, his absolute error would also be 1.5 percentage points (overestimates and underestimates are treated the same).
After calculating the absolute errors for each forecast in every quarter, we can take the average of the absolute errors from all forecasters in a particular quarter to gauge the forecasters’ overall performance in that quarter. Then, we can look for two things: first, absolute performance in every quarter, to answer the question of how well economists perform at macroeconomic forecasting, and second, change over time, to answer the question of whether economists’ macroeconomic forecasting has improved over the decades.
The SPF dataset includes 6,808 forecasts of unemployment and 6,716 forecasts of nominal GDP. Here, I will only examine forecasts that are made four quarters ahead (for example, forecasts that are made in 1975, quarter 2, about the final realized unemployment rate in 1976, quarter 2).
The mean absolute errors over time for SPF predictions of U.S. unemployment are shown in the chart below. In this figure, the black dots show the quarterly mean absolute errors of unemployment forecasts. So a value of 1.5, for example, means that the average forecaster’s guess was 1.5 points away from (above or below) the true value: either guessing 4.5 percent or 7.5 percent unemployment when actual unemployment was 6.0 percent. The solid line shows a “Loess curve,” which is something like a rolling average of the plotted mean errors. (A Loess curve is based on a weighted local regression methodology rather than a raw average.) The dotted lines show a “confidence interval” for the solid Loess curve.
One of the easiest things to notice about the errors plotted in the above chart is how large they get. The majority of quarterly average errors are larger than half a percentage point, which is a considerable difference for a figure that is almost never outside the 6-point window between 4 and 10. In many quarters, the average error is larger than 1 percentage point and even exceeded 3 points in one quarter. The value of the Loess curve is never less than 0.5 percentage points.
As for the trend of the data, there is no sharp increase or decrease. There are many different methodologies for finding time trends in data, and some of these methods will indicate a slight downward trend for these errors. At best, the downward trend is very slight. However, there is evidence that the errors are stagnant:
- The Loess curve reached its minimum value in 1996, two decades ago. Throughout the 2000’s the Loess curve rises, as does its confidence interval.
- The most recent confidence interval contains the estimated Loess value from the earliest recorded quarter (1968, quarter 4). Similarly, the earliest confidence interval contains the estimated value from the most recent recorded quarter.
Another important thing to point out in this data is that the errors appear to be especially large just before and during recessions. We can see spikes around 1973–1975, in the early 1980’s, and the early and late 2000’s, all periods of recession. These spikes are similar to each other in maximum height. The period of best forecast performance was the 1990’s, a period of steady and sustained economic growth.
I believe the above figure is damning to the economics profession. It shows a profession that frequently makes large errors (see the many high error values plotted), that predicts recessions very poorly or not at all (see the bad performance around recession quarters) and that has improved little or not at all over the years (see the relatively flat and sometimes-increasing “average” line). The errors around recession years are consistent in their large size—I cannot see evidence for an increased ability to predict recessions. Looking at the chart, it is easy to wonder whether the hundreds of economics research papers published annually are worth the millions of dollars that are spent funding them. It is not clear that they deliver anything more than a flat Loess curve indicating stagnation. The yearly Nobel Prize in economics is meant to represent game-changing advances in the field. But it is not clear that big advances have happened at all in economics over the time plotted above, let alone the annual game-changing advances that the Nobel committee claims.
So, economists have a poor track record at prediction, and show little or no evidence of progress over the decades. If they cannot predict well or improve at prediction, it is not clear why we should believe that they have done any better at anything else they claim to be able to do. It is also not clear that we should continue to ask their opinions about the economy’s state or future. Neither is it clear from these poor results that we should exalt them to cabinet posts or give them column space in prestigious periodicals. What then is their value? I will consider this below.
Interpreting the Data
Before moving on to assess the value of economists in public life, it would be worthwhile to consider the implications and limitations of the data shown in the case study above. The SPF dataset is well-suited to an evaluation of the performance of economists for several reasons. First, simply because it consists of methodically collected data rather than anecdote, it rises above the anecdotal cases frequently made by pundits about economic issues. Second, because the survey has asked the same questions in the same way across its entire existence, it enables comparisons between different time periods that would otherwise be hard to compare, including times of peace and war, boom and bust, low interest rates and high, and a variety of different types of elected government administrations.
A third and very important advantage of the SPF as an evaluation metric for economics is that it provides a pure measure of the performance of economists and their knowledge rather than the performance of the economy itself. Evaluating economists based on the performance of the economy always runs into the impossible challenge of finding the right counterfactual as a reference point. In other words, any criticism like “you mismanaged the aftermath of the Great Recession and all of your policies led to a sluggish recovery” can be met with the response “the recovery was indeed sluggish, but it would have been much more sluggish without my perfectly wise and effective policies.” There is no obvious reference point for performance, so 3 percent growth could be argued to show poor management by economists since it is less than 5 percent growth, and 2 percent contraction could be argued to show good economic management since it is better than 4 percent contraction.
No matter how bad the performance of the economy, economists can cite an imagined world without them full of recessions to justify their existence and high salaries. With SPF data, by contrast, the 0 lower bound of absolute errors is indisputably the right reference point. No economist can argue that his 5 percent error rate was perfect because it was less than 6 percent – it is clear to everyone that 0 percent error is the right place to be.
A potential objection to this case study has to do with the validity of the inference from macroeconomic forecasting to the rest of economics. An economist might concede that macroeconomic forecasting is a relatively stagnant sub-field of economics, but argue that its poor performance does not reflect on the performance of microeconomics, or game theory, or optimal taxation theory or some other corner of economics.
This is a reasonable objection since it is true that macroeconomic forecasting does not comprise the entirety of economic science. However, there is no reason to suppose that forecasting progresses any slower or faster than the rest of economics. Research about macroeconomic forecasting appears in top economics journals alongside research about other topics in economics. Entire journals exist that are solely dedicated to forecasting techniques, and some respected economics scholars dedicate entire careers to advancing the forecasting sub-field. To my knowledge, no top economist or Federal Reserve chairman has ever expressed a belief that economists are unable to make accurate forecasts, that forecasting is not part of “true” economics, or that our ability to forecast progresses more slowly than our other economic knowledge or abilities. Macroeconomic forecasting is as good a proxy for the economics profession as any.
In fact, a stronger case can be made. Instead of viewing forecasting simply as a proxy for the rest of economics, we could view it as a foundation for everything else that economists do. If a doctor cannot accurately predict what effect some medicine will have, should we trust the doctor to prescribe it? If the local authorities cannot tell when hurricanes are coming, should we evacuate our homes when they start making noise? In these examples, the ability to predict is a prerequisite for the basic trust required to listen to advice or follow prescribed treatments. This view is consistent with Karl Popper’s falsification paradigm of science, in which verification of novel predictions is the only method for deciding whether to accept or reject theories. From this perspective, the large errors and general stagnation of economic forecasting accuracy is one sign that the entire field has performed poorly and failed to progress for many years, and that the prescriptions of economists are therefore not trustworthy.
The Proper Role of Economists in Public Life
I have argued that economists cannot accurately predict the future, that their field has been stagnating for decades, and that we should not therefore blindly trust their prescriptions for healing a suffering economy. But saying all of this is not to say that economists have no value to provide to society. Though they are not equipped with the knowledge and tools necessary to predict and eradicate all economic “plagues,” they do not need to be dismissed entirely as quacks.
I suggest a more fruitful analogy than the economist-as-doctor idea: I suggest that economists can have a role in society that is comparable to the role of geologists who study earthquakes. Earthquakes can be of great importance: they destroy whole cities, cause tsunamis and fires, bring down strong buildings, and can harm many people. But geologists, despite all their enormous expertise, do not usually make pretensions of abilities beyond what they possess——they do not claim to be able to control or stop earthquakes, or even to predict their occurrence or strength with much accuracy. Earthquakes, as much as we have progressed in our understanding of precisely what causes them and how they happen, are beyond human control or prediction, and geologists recognize this. Partially as a result of the geology profession’s humility, we do not have a highly-paid presidential council of geologists who give advice about how to ensure the health of our plate tectonics and avoid earthquakes in the way that we have national councils of economists giving advice about how to avoid recessions.
However, earthquake experts can still provide great value to society. I believe that our method of confronting economic recessions could benefit from emulating our method of confronting earthquakes. Since we cannot know exactly when and where earthquakes will strike, we simply build all buildings with sufficient strength and engineering to withstand the largest earthquake that is likely to occur at the building’s location. We take out insurance policies on man-made structures in case disaster strikes. We draw up simple, clear evacuation plans and emergency procedures for people to follow if needed, far in advance. We have government departments, not to mention private charities and organizations, that are prepared to be “first responders” to arrive on scene and starting picking up the pieces even before the dust settles.
Geologists can be useful not by controlling earthquakes or even telling exactly where and when they will occur, but by providing more general advice about how common strong earthquakes are, how strong they are likely to be, and how to prepare for their worst effects. They can consult with engineers about how and how much to fortify a planned building. They can help insurance actuaries decide on the right risks and insurance rates for buildings that get insured. They can help city planners decide where and when to allow different types of building permits to be issued based on local risk profiles. They can provide some very limited predictions minutes or hours before disasters. After the start of an earthquake, they can guide the emergency response, especially by providing insights into secondary effects (like where and when a tsunami may hit land).
If we treated the economy in the same way that geologists treat plate tectonics, we would have a humbler and more robust approach. Instead of trying to avert and control inevitable recessions, we would focus on preparing and learning to deal with economic phenomena as if they were acts of God or nature—inherently unpredictable and uncontrollable. In some limited cases, it may be possible for the economics profession to assist the private sector or even governments in insuring or hedging against recessions. Economists could also help by drawing up simple, clear “first-response plans,” like we have for natural disasters, including recommendations for what to do on day one of the next inevitable recession. Without claiming control or special prescience with respect to the economy, economists could describe in general how we might expect recessions to look, to help us be better prepared.
The current, counterproductive approach of the economics profession is borne of our arrogance about how much we can understand, predict, and control. We appoint top economists to government posts not primarily to prepare for the fallout of disasters, but to somehow control things so that recessions never happen in the first place. This is an impossible dream. And we do not have good plans in place for what to do when another recession inevitably strikes—rather, we are caught unprepared for every recession and have the same old arguments about Keynes, stimulus, and the rest, each time. We do not insure ourselves against economic disaster, thinking that our latest crop of economists in charge will make insurance unnecessary by making disaster a thing of the past.
Treating the economy like the earth’s crust makes sense when we consider the complexity of human psychology. Psychologists, for all their sophistication and advanced knowledge, cannot perfectly predict any single individual’s behavior. No CAT scan or psychological evaluation can know with certainty exactly what a person’s “consumer confidence” will be tomorrow, or exactly which products he will buy or exactly how much he will save or invest or create or steal. If we cannot predict these things accurately for a single person, how can we expect to be certain about them for millions of people? True, studying aggregates rather than individuals makes some things easier because one need only predict averages and extremes will cancel out. On the other hand, jumping from studying one person to studying a whole nation also introduces crowd effects, externalities, mob mentalities, contagions, and in general enough complexity to make economic prediction and control somewhere between daunting and impossible. For as long as complete mastery of individual psychology remains outside our grasp, perfectly predictive macroeconomics and an ability to positively control national economies will remain no more plausible than the dream of perfectly predicting and controlling earthquakes.
The suggestions I have made for economists’ role in society do not require a research-industrial complex spending millions of dollars every year on writing new academic papers on esoteric topics. I believe that the complexity that economists create has lined the pockets of economists who set themselves up as the only people capable of understanding it. However, see the chart above——all of the complex models that economists have created for the last decades have had only a negligible impact on our economic abilities. Until economists’ complexity can prove its worth, I suggest that the national approach to economics can be simple and inexpensive and just as effective.
Incoherence and Its Discontents
Above, I have argued that economics has performed poorly, that it has failed to progress, and that its performance has been the worst around the times of recessions, when it is most crucial that it be accurate. I will go a step further in my indictment of the economics profession and say that economists have been incoherent.
The incoherence of the economists can be seen in the profession’s failure to live up to its own standards. Consider: when there are discussions of public programs or policy changes, economists argue for more data, for quantitative assessments of impact, and for decisions based only on hard-nosed calculations of net expected benefits. However, if anyone ever questions the efficacy of government-funded research economics, councils of economic advisers, or enormous and expensive academic institutions dedicated to economics, economists lose their dedication to impartial quantitative calculations and resort to bare rhetorical assertions. This was evident recently when there were rumors that President Trump would demote or even abolish the Council of Economic Advisers (CEA). Media outlets and economists were uniformly aghast at the idea, and denounced it in no uncertain terms. Tellingly, their arguments for keeping the council were not based on economic ideas or the kind of quantitative analysis that characterizes the field. Instead, the arguments in favor of the CEA were notably empty and ad hominem——claims that Trump was a reckless idiot rather than numerical assessments of the value that the CEA is expected to provide.
The reason that economists did not argue for the Council of Economic Advisers in economic terms is that the economic evidence for the efficacy of economics is scant and quite weak. It is extremely rare for an economist to use the tools of economics to find the value that professional economists add to the economy, and so far, none has made a slam-dunk economic case in favor of the field. The case study I have presented above constitutes an attempt to use the quantitative methods typical of economics to argue against the field. The central incoherence of professional economics is that while they argue for rigor and serious economic study of every public policy and expenditure, they have not been able to make a rigorous economic case for the policies and expenditures that enable their own existence and prosperity.
There are other cases that could be made to argue for the incoherence of the economists. If one has a chance to read applications for grants and funding or the more ambitious research paper abstracts, there is a logical disconnect between stated goals and achieved results in economics. In grant applications, researchers describe themselves as constantly just on the brink of a serious breakthrough—that if they just had a little more research funding, they could settle some very important question once and for all. But retrospectively, the ratio of serious breakthroughs to grant applications is so vanishingly small that these claims could not possibly be taken seriously.
Of course, this incoherent disconnect between goals and achievements is not unique to economic research. But the track record of economics is quite poor: for example, since the beginning of the Great Recession, there have been renewed and vigorous arguments about various ideas attributed to Keynes (who died in 1946). How could it be that, many decades after his death, we still have achieved so little consensus about whether and how to implement his ideas? The same is true for such foundational ideas as the minimum wage, a very old idea taught in Economics 101 whose value as an idea still has no clear consensus among professionals. How can economists write grant applications describing alleged upcoming breakthroughs, when we cannot even decide the truth or falsehood of foundational ideas like the minimum wage that are many decades old and have been studied by thousands of researchers funded by millions of dollars? How many more decades and fortunes will we have to spend to get to the truth about minimum wage, Keynesian stimulus, or anything else in economics? It is time to apply quantitative analysis and utilitarian reasoning (the tools of economics) to a field whose ambition and self-promotion have outstripped its accomplishments.
Managing our Household
The economics profession has been guilty of extreme intellectual overreach. The best economists have impressive skills in mathematics and statistics, including the difficult linear algebra that underlies economic estimation methods. Most top economists also have skills in programming and logic. This is not even to mention the political skill required to coordinate economic research projects, the intuition required to find the right research questions to ask, and other skills that are common among the best economists. However, these important and impressive skills are not enough to give professional economists the final word in the biggest policy questions——or at least they should not be.
In the 2016 debate about Brexit, we saw overreach as economists pointed out (their belief) that Brexit would have a negative impact on Britain’s economy. This was a reasonable point to make; the overreach came when economists and pundits assumed that by “proving” that Brexit was economically unwise, that it was also ipso facto the wrong choice to make. The assumption underlying this line of argumentation is that the economic yardstick is the only or most important one by which we can judge decisions. The negative economic consequences of Brexit, even if we stipulate that they are as bad as the most pessimistic economists feared, are only one among many things that should be taken into account when deciding to preserve or dissolve political bonds. After all, just because there is an economic price to pay for something doesn’t mean that the goal isn’t worth the price. Patriotic Americans could reflect on the negative economic consequences of “Amexit” and how the Founders still thought it worth pursuing, even if it was sure to make them a little poorer economically. Somehow, between then and now, we have shifted our thinking to suppose that economics matters more than independence——some think it matters more than anything else.
Edmund Burke saw this coming, just a few years after the Revolutionary War. In one of the most famous passages in his Reflections, he lamented the advent of the age of economists: “But the age of chivalry is gone. That of sophisters, economists; and calculators has succeeded; and the glory of Europe is extinguished forever.” The passage from which this is lifted describes the bygone world that Burke missed, a world that Burke described as containing things like “generous loyalty,” “exalted freedom,” “unbought grace,” “gallant men,” and “honor.” Nowhere does he lament the loss or praise the gain of a few percentage points of GDP, of affluence or technology or efficiency. Burke saw that the glory of Europe was not in those things, but in the nobler feelings of the heart, in the pure and humane relations between people, and in the culture and civilization that his forebears had bequeathed to his generation. In his sympathy for “Amexit” and his criticism of the French Revolution, he was motivated scarcely at all by economic considerations, but rather by considerations of other cultural treasures that he regarded as the “glory of Europe.”
This is not to deny that we should expend great effort trying to increase GDP, calculating the best decisions for our economy, and learning to mitigate the worst effects of recessions. But as we do those things, we should remember that the economic considerations should always be secondary to other, more important things. What are those more important things? A story about an exchange between the economists Wilhelm Roepke and Ludwig von Mises provides an excellent answer:
During the Second World War, the city of Geneva had made available to its citizens plots of ground along the ring around the city where the ancient walls had stood. On these allotments, in time of scarcity of food, the people of Geneva, particularly the laboring folk, could cultivate vegetables for themselves. These allotments turned out to be so popular, both as recreation and as a source of supplementary food, that the city continued to make this land available to applicants after the war was over.
Now Mises, who had been professor years before at the Geneva Institute of International Affairs, came to visit Roepke in Geneva, about 1947. Happy at the success of these garden allotments, Roepke took his guest to see Genevan working people digging and hoeing in their gardens. But Mises shook his head sadly: “A very inefficient way of producing foodstuffs!” he lamented. “Perhaps so,” Roepke replied. “But perhaps a very efficient way of producing human happiness.”
Producing food efficiently is a wonderful thing—but food is not an end in itself. The efficiency of food production is at best an abutment to efficient human happiness production. Humane and humble economists like Roepke have always understood that the efficiency and production they spend their lives studying is usually only secondary to more important human concerns. Burke would have sided with Roepke rather than von Mises in this case—thinking more of the freedom and culture that were cultivated in the gardens than the wealth.
It is worth remembering that the management of a household is the root of economics——the word “economy” itself comes from the Greek “oikonomia,” meaning household management. There are matters of money and finance that are settled within a household, but the most important process in any household is the creation and development of people: humans who are most poised to experience happiness. As in the family household, so in the national household: we should be more focused on creating the conditions necessary for the flourishing of free people than on the more mundane and less ultimately important matters of penny pinching and maximizing profits.
Economists would do well to realize that for all its apparent simplicity, a human household still contains many mysteries that scientists, social or otherwise, have not fully understood. The rearing of souls from birth to self-reliant adulthood and the creation and maximization of real happiness—not just contentment or wealth or distraction from pain—are difficult and mysterious processes. If we are to ever understand them in a scientific way, we must take one little step at a time. The flow of money is just one small element of these household arts, and even that element is not fully understood by economists.
The economics profession has much good to offer our society. My aim here is not to deny this. Rather, by suggesting a role for economics that better fits its scientific capabilities, I hope to point the way towards a better future for economists and non-economists alike. After all, we are all members of the same national household.